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The  purpose  of  this  paper  Is  to  review  recent 
progress  In  the  actual  construction  of  optimal  non- 
linear filters.  Specifically,  we  will  consider  the 
construction  of  numerical  algorithms,  which  accept  as 
Inputs  noisy  observations  of  a nonlinear  function  of 
signal  process  and  produce  as  outputs  estimates  of  the 
signal  process.  These  algorithms,  because  of  the 
structure  of  the  nonlinear  filtering  problem,  can  be 
thought  of  conditional  probability  function  generators. 
A historical  review  of  the  theory  and  applications  of 
nonlinear  filtering  will  be  given,  which  will  attempt 
to  catalog  the  seminal  Ideas  (n  the  field  as  well  as 
some  unsolved  problems  which  are  obstructing  progress. 

1.  Introduction 

History  of  the  Theoretical  Resolution 

The  theory  of  nonlinear  filtering  was  initiated  by 
Stratonovlch  in  [1]  In  1960.  He  assumed  that  the 
signal  x(s,w)  was  a Markov  diffusion  process  taking 
values  In  (A,  and  further  that  observations  were  avaiV 
able  of  the  form 

r t 

z(t,w )*/  h(s,x(s))  ds  ♦ v(t,w)  (1.0) 

Jto 

where  v(t,w)  is  a Brownian  motion  process  Ks  valued 
with  Infinitesimal  covariance  «(t)  and  h is 
measurable  function  from  (to.  -)  « ft"  to  n%.  The 
problem  consisted  of  finding  the  conditional  distribu- 
tion of  x ( t ) given  z(s,w)  for  t«  «.  s «.  t,  as  if 
one  Is  Interested  In  estimating  x(tj  on  the  basis  of 
knowledge  of  the  observation  sample  function  z(',w), 
this  conditional  distribution  contains  all  the  infor- 
mation relevant  to  the  estimation  process.  Let  us 
assume  that  x(t)  possesses  a transition  density 

p(t,x,y)  (i.e.,  P(x(t)  t A|x(o)  * t) 

*/  P(t,i,y)  dy  1 
J A 

and  further  the  following  Llndeberg  conditions  hold 
E(x(t+h)  - x(t))|x(t)  ■ y)  ■ f(y)h  + o(h)^ 

e(  (x(t+h)  - x(t))(x(t+h)  - x(t)  )*|x(t>  * y) 

* o(y)  Q o'(y)  h+o(h) 

Under  these  conditions  and  smoothness  Kolmogorov  in  [2] 
demonstrated  that 

|f  ■ Ap  (1.2) 


so-called  backward  equation.  The  equation  defines  the 
evolution  of  the  expectation  of  functions,  the  condi- 
tional density  is  a’solution  of  (1.1)  with  Initial 
value  p(o,x,y)  « 6(x-y),  or  In  other  words  p(t,x,y) 

Is  the  Green's  function  of  (1.1).  If  on  the  other  hand 
the  Initial  value  Is  k(y),  the  solution  Is 
E k(x(t))|x(o)  « x)  . A dual  equation  was  also  derived 
as 

ff  * A a (1.3) 

where  A is  the  formal  adjoint  of  A and  the  general 
solution  of  It  is 

*(t,x)  * Jp(t,z,x),(z)dz 


or  probabilistically  >(t,x)  Is  the  density  of  x(t) 
when  x(o)  has  density  u.  Notice  that  p( • , • ,y ) 
satisfies  (1.1)  while  p(>,x,*)  satisfies  (1.2). 
Suppose  k(x,t,z.)  is  the  conditional  density  of  x(t) 
given  z(s,w)  t0  < s * t , then  Stratonovlch  showed, 
using  Bayes  rule  an?  truncating  a Taylor  series,  the 
equivalent  of 

dk  « A kdt  ♦ (h-ht)'R-1  d 1 k (1.4) 

where  (1.3)  is  a random  differential  equation  of  Ito 
type , see  (3).  The  initial' condition' of  (1.4)  Is  the 
density  u.  of  x(o),  and  fu  */h(t,y)  k(t,y)  dy  with 
dl  ■ dz  - h dt.  In  (6|,  Ito  showed  that  if  (1.1)  holds, 
f and  o are  Lipshitz  and  then  x(t)  Itself  is  the 
solution  of  the  Ito  equation 

dx  » f(x)dt  ♦ o(x)da  (1.5) 


where  8 is  a Brownian  vector  process  independent  of 
x(o)  with  infinitesimal  spectral  matrix  Q. 

Now  (1.4)  is  locally  a description  of  the  nonlinear 
filtering  problem,  and  since  In  Quantum  Mechanics  the 
local  description,  the  Schroedinger  equation, has  a - 
global  analog  the  famous  Feynman  Path  Integral,  see{38) 
and  139),  one  might  ask,  does  the  nonlinear  filtering 
problem  possess  a global  description?  In  fact,  this  Is 
the  case,  although  it  was  not  until  1965  In  (7)  that 
this  global  description,  The  Representation  Theorem, 
was  proposed.  Assuming  regularity  conditions,  see  [8] 
through  |14],  for  the  details1* 

k(t,x)  * - x).p(x)  (M) 

Ez  • e"t 


where  E • means  average  with  the  observation  path 
fixed  and  t ( t z 

mf  h(s.xs)Rs  at  ~ ? ft  I !h(s,xs)  ||  .)  ds 
o 0 Rs 

11.7) 


It  is  interesting  to  note  that  the  conditions  for  the 


*Th1s  research  was  supported  In  part  by  the  United  States  Air  Force,  Office  of  Scientific  Research,  Air  Force 
Systems  Command,  under  AFDSR  Grant  71-2144. 

zThe  order  within  the  trace  Is  Important  here. 

Actually,  In  (1)  another  form  of  (1.3)  Is  given  which  requires  to  be  valid  in  Interpretation  In  terms  of  a 
« Integral  detailed  In  (4).  Kushner  In  (5)  was  the  first  to  derive  (1.3)  In  the  Ito  form.  See  also  (33). 

6Mortensen  In  (9)  first  recognized  that  the  representation  theorem  was  In  fact  derivable  from  the  chain  rule 
fine  Radon-NIkodym  derivatives  in  function  space;  the  most  elegant  proof  so  far  Is  given  In  [12J,  where  a Hilbert 
setting  reduces  the  nonlinear  problem  to  a linear  one,  where  the  result  Is  easy. 

App-jved  for  public  rel#Mt{ 
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validity  of  (1.6)  are  condition*  on  the  signal  process 
and  the  sensor  or  conditions,  not  a priori  unverifiable 
conditions  on  k such  as;  k be  twice  continuously 
differentiable  which  Is  necessary  for  (1.4)  to  be 

Vt  1 id. 

A largely  heuristic  approach  to  nonlinear 
filtering  theory,  the  so-called  Innovations  approach, 
discovered  by  Frost  in  (15)  and  popularized  by  Kallath 
and  Frost  in  a number  of  papers  (see  (16)  for  refer- 
ences),hinges  on  transforming  the  observations  to  pro- 
duce a new  observation  process  that  is  white  and 
consists  of  “new  information"  at  each  instant, 
generalizing  ideas  of  Kolmorogorov  in  1 1 7 1 . While 
these  ideas  am  clearly  useful,  a number  of  results4 
claimed  have  yet  to  be  proven. 

It  seems  convenient  to  consider  here  the  discrete 
sequential  version  of  the  representation  theorem  as 
for  numerical  purposes  it  seems  the  rost  usefu1--see 
(17)  for  an  early  occurrence  of  this  result.  Suppose 
both  z(t,w)  and  x(t,w)  are  sampled  with  sampling 
interval  a and  denote 

xn  * x(n  a ♦ t0,w) 

zn  - z(n  a*  tQ,w)  -z((n-l)  A ♦ tQ,w) 

Farther,  suppose  that  the  conditional  density  of 
*0*1  1 y 9’ven  xn  = x is  Sn(y,x)  and  the  condi- 
tional density  of  zn  given  xn  = x is  D„(x,zn) 
then 

pn*-l(y)  =/(  Sn(y,x)  Fn(x)  dx  (1.8) 

F„(x)  • y„  On(x.z„)  Pn(x)  (1.9) 

w!  ere  PnfFp)  are  respectively  the  conditional 

deisitiesof  x„  given  zn.j  zi,  iz„,  zn_|  . . . zj ) , 

and  yn  is  the  appropriate  normalizing  term.  Note 
that  (1.8)  represents  >_odel  following  while  (1.9)  re- 
presents the  influence  of  the  new  piece  of  data,  the 
analog  of  the  contributing  factors  of  estimate 
dynamics  in  the  linear  case. 

2 . Problems  Arising  in  Numerical  Pe  lization 

Let  us  note  that  in  continuous  tire  both  the  local 
and  global  dynamics  of  the  conditional  density  (1.4) 
jod  (15)  involve  a non-poinfwise  limiting  process, 
specifically  a limit  in  the  mean  because  of  the 
definition  of  the  stochastic  integral --see  (6).  In 
vi«w  of  this  finding,  the  value  of  k(t,x,z.)  when  a 
sample  path  z(*,w)  is  given  by  direct  difference 
approximation  of  (1.4)  or  replacing  x(t,w)  in  (1.6) 
by  random  process  which  has  at  most  finite  number  of 
values  for  each  w can  lead  to  divergent  approxima- 
tions negative  values  for  the  approximations  to  the 
density  k(t,x)  in  the  case  of  (1.4)  and  in  general 
disasterous  numerical  behavior. 

It  is  also  a problem,  illustrative  of  our  last 
remarks,  to  produce  numerically  the  continuous  time 
white  noise  processes  sample  functions,  in  fact,  it 
was  shc-wn  by  Wong  and  Zakai  in  (181  that  the  solution 
of  the  scalar  stochastic  differential  equation 

dx  = f(x)  dt  + o(x)  d3 
is  in  general  different  from  the  limit  of 

- f(xn)  ♦ «{*„)  W„  (2.1) 

where  Wn  is  the  derivative  of  an  absolutely 


continuous  function  and  such  that 

s(t.w)  - 3 ( t ,w)  « limj^wn(s)  ds 
in  fact,  x*  « lln  xn  satisfies 

dx*  » f(x*)dt  ♦ (x*)dt  ♦ u(x*)d9  (2.2) 

dx 

On  the  other  hand,  numerous  procedures  for  computer 
realization  of  approximate  white  noise  sequences  exist, 
although  most  of  them  are  fairly  poor  approximations, 
especially  the  canned  subroutines  available  for  the 
IBM  and  CCC  '-achires,  and  most  of  the  others  pass 
statistical  tests  which  depend  fundamentally  on  their 
assumed  ergodicity.  In  (19),  Senne  develops  a 
generator  which  is  not  only  machine- independent  but 
further  passes  the  Kolmogorov-Smi ronov  test  for 
distributional  fit.  For  all  generators  judicious 
choice  of  the  se»d  is  important. 

It  appears  then  that  it  is  preferable  to  sample 
both  the  signal,  x(t,w)  and  z(t,w)  at  a rate  faster 
enough  not  to  lose  information  relative  to  the 
continuous  problem--see  (23)  and  (211  for  an  analysis 
which  determines  the  sampling  rate  for  the  phase 
demodulation  problem--  and  to  use  (1.8)  and  (1.9)  to 
realize  th»  nonlinear  filter. 

Another  numerical  problem  is  that  in  order  to 
evaluate  filter  performance,  Monte  Carlo  runs  must  be 
performed.  This  requirement  taxes  the  ability  of 
modern  third  generation  digital  computers  for  problems 
with  low  state  dimensions  signal  processes.  Further, 
the  number  of  Conte  Carlo  repetitions  must  be  large 
enough  to  provide  small  enough  confidence  bands  on  the 
error  performance  so  that  the  optimal  filter  per-  5 
formance  can  be  meaningfully  compared  with  sub-optimal 
filters--see  (22).  Hopefully,  more  research  on 
a priori  bounds  will  eliminate  the  need  for  costly 
Monte  Carlo  simulations.  Pronising  research  in  this 
direction  is  reported  in  (23). 

Finally,  I think  it  is  appropriate  at  th.s  point 
to  indicate  why  it  is  ’-'portant  to  undertake  numerical 
realization  studies.  Primarily  these  studies  are 
important  in  order  to  conclusively  demonstrate  the 
degree  of  superior  error  performance  which  can  be 
achieved  using  the  optimal  nonlinear  filter.  A sub- 
sidiary hene'it  is  that  insight  is  ga>ned  on  the 
behavior  of  nonlinear  filters.  Because  of  a paucity 
of  examples  wh ere  closed  form  solutions  exist--see[24) 
for  a numbe"  of  such  e<amoles--there  are  few  opportu- 
nities to  check  conjectures  as  well  as  to  gain  insight 
into  what  properties  might  be  generally  true.  Without 
examples,  the  field  of  nonlinear  filtering  could 
easily  develop  into  a stale  effete  area  wnich  dies  by 
feeding  on  itself  and  is  overburdened  by  work  which  is 
neith®r  good  mathematics  nor  useful  engineering. 

3 . Conditional  Density  Representation 

It  is  clear  that  for  digital  computer  iteration 
of  (1.8)  and  (1.9)  a rap  T from  a subset  L of 
(o,l 1 X to  a finite  dimensional  vector  space  K must 
be  given--here  y is  a subset  of  .id.  if  x is 
compact,  the  map  can  be  fixed,  while  if  x is  not 
compact,  the  rap  rust  change  with  time.  Some  examples 
will  clarify  the  general  idea. 

Example  1 (see  (251  ) 

L 1 C0(  — ,*)  , d = 1 


Specifically  the  proof  of  the  innovations  theorem  in  (151  is  wrong. 

‘lAll  variables  can  take  vector  values  and  the  Integral  may  be  rulti-dipensional . 

7Cr.  Senne  informs  me  that  the  generator  has  been  realized  on  an  H.P.65  programmable  hand-calculator,  so  that 
base  2 assumptions  in  (19]  are  unnecessary. 

“Notice  that  generally  error  performance  of  sub-optimal  filters  riust  be  evaluated  by  Vonte  Carlo  methods. 
Further,  the  statistical  design  of  the  Monte  Carlo  trials  mustallow  for  the  ronergodic  nature  of  the  error. 
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<Tn  PnHO  * {pn<*?>lM  ^ *„(«> 

is  the  average  of  Pn  over  « bil l-centered  at  x. 

xl"  ' “n|n-2  + “Sfer  (i  ' M * 11 
where  H is  an  integer  and  un| „_2  an<l  nn|n-2  ir* 

“n|n-2*  * xJzo  ••••zn-l) 

°n|n-2  ’ ^xn  lzo""zn-1^  * 'n|n-2 

and  the  ' superscript  denotes  averaging  with  respect 
to  « 2MH  . „ , n-1 

Cl  <Tn-1  Pn-l><x}  * jl,  Pn-1<C  ’ ,(mJ  h 

TZ  ^ denote  a choice  of  pre-image  of  an  element  in 
the  range  of  Tn_p 

Example  2 d * 2 (see  |29|  ) 

|T"  "»>  ■ 

Pn(x,y)  denotes  the  average  of  Pn  over  a ball  of 
small  radius  centered  at  ( x ,y ) 

^y‘„^  * «fi|n-2  + »'t  * Vth  (■l-t‘-’>?2n 

M and  H are  integers 

>(nf  ej0  are  eigenvalue  and  eigenvectors  of  ^njn_2 
where  . 


Example  6 (see  (20),  |26)  ) 

L is  the  set  of  functions  on  the  Torus  in  ffi 
and  T assigns  to  a function  its  values  on  a uniform 
grid  of  meshes 


in  each  coordinate. 

Example  _7  (see  (30)  ) 

The  map  T assigns  to  a function  a finite  subset 
of  its  non-interpolative  spline  coefficients. 

Example  8 ( see  (311  ) 

The  rap  T assigns  to  a function  coefficients  of 
a least  squares  or  1‘  fit  of  the  function  to  a finite 
linear  combination  of  functions. 

In  all  nf  these  cases  (1.8)  and  (1.9)  are 
approximated  for  synthesis  purposes  by  the  vector 
matrix  recursion  relation 


Jn  1 K<">Jn-l 


where  Jn  is  the  image  of  either  Fn(x)  or  Pn(x) 

under  1 and  1 indicates  that  J must  be  re- 

n n 

normalized  nr  transformed  so  that  a canonical  choice  of 
pre-image  ot  J , which  we  denote  by  T*  Jn,  is  a 
density.  The  relation  (3.1)  can  be  arrived  at  in  the 
following  way.  first,  one  notes  that  in  the  one-step 
predictor  case,  for  example 

rn*l(x)  “ 'n  / S(,>v)  nn<y,z">  V*’  d* 

* 


>.(n,  e ^ are  eigenvalue  and  eigenvectors  of  Snjn_?  vR  / :(>.y)  Dn(y,zn)  T*  Tn(Pn(v))  dy 

wh§r*  , r • . , • and  applying  T . to  both  sides,  it  follows  that 

Sn|n-2*  ' xn  *n  |zo zn-2>  ' i;n|n-2%!n-2  n° 

1 -.V.  which  Is  equivalent  to  (3.1).  fl  problem  which  leaos  ':z 

and  - supersenot  denote  averaging  with  respect  to  the  numprjci)1  instability  is  The  following;  suppose  (T  . 


density 


X’  X Pn-1  yj",)  ',(x*Xi*1)  ^x-y^'1). 

Example  3 l is  the  set  of  continuous  probability 
densities  on  the  Torus 

TV  K,.q--‘,),  , 

1 z rMt.  ‘ii. 


and  {TM  are  r.hosen  and  tHe  relation 
n 

J„  • K(n)  Jn_,  (3.2) 

t 

is  iterated,  the  sequence  Tn  Jn  does  not  always 
remain  positive,  pven  when  nJ0  is  a vector  with 
T*  J0  positive.  In  Example  5,  a convenient  and 

effective  modification  of  (3.2)  to  preserve  positivity 
is  redefining  Jp  as 


■„(i)  * max  (o,(K(n)Jn1 ) (1 )) 


In  examples  1,  2 and  6,  Tn  can  be  chosen  so  that  the 
above  negativity  effect  does  not  arise. 


H,  are  integers,  and  a?  . are  Fourier  cneffl-  t 

J r r In  examples  1,  2 and  6,  Tn  can  be  chosen  so  that  the 

cipnts  of  Pfl  (see  (27)  ).  above  negativity  effect  does  not  arise. 

Examolo  4 (see  (28)  ) For  problems  where  the  signal  process  is  a 

— — 1 , degenerate  random  process  (i.e.,  a random  variable) , 

The  map  T assigns  to  a function  a finite  subset,  repre5entation  theorem  gives  an  explicit  expression 

its  Interpolate  spline  under  tension  coefficients.  for  the  conditional  density  and  the  problem  of  re- 

presenting the  density  is  trivial--see  (32)  for  results 
Example  5 (see  (49)  ) concerning  this  degenerate  case. 

The  rap  T assiqns  to  a function  a finite  subset  Thepoint  mass  representation,  Examples  1 and  2, 

of  its  coefficients  in  a Causs-Hermite  expansion.  was  the  first  one  considered  and,  in  fact,  can  be  made 

’The  choice  depends  on  whether  one  wishes  to  synthesize  the  filter  or  one-step  predictor. 


j 


quite  accuracy  by  Increasing  the  number  of  grid  points 
until  the, signal  estimates  for  a fixed  sequence  of 
observations  agree  to  say  four  places  by  successive 
choices  of  finer  subdivisions  of  the  grid,  the 
accuracy  obtained  by  this  method  Is  not  quite 
unexpected— see  for  example  the  discussion  on  Page  4 of 
1341,  where  coincidence  of  form  is  compared  with 
metric  closeness.  The  drawback  of  the  point  mas?, 
rethod  consists  of  the  large  computation  time  per 
estimate,  and  in  fact  the  other  representation 
methods  were  motivated  by  the  desire  to  decrease  this 
estlrate  time,  while  preserving  a given  accuracy  re- 
lative to  a point  mass  accuracy  benchmark.  A some- 
what different  approach  to  the  representation  problem 
consist  of  determination  of  a perturbation  series  for 
Pn  and  Fn  in  (1.8)  and  (1.9)  when  Sn(y.x)  depends 
on  a parameter  q;  for  example,  suppose 

. (i-J :1\ 

Sfx.y)  • 4=  e 

' 2*q 

then  Fn  and  Pn  can  be  determined  as  series  In  q, 
see  (35)  and  (36)  for  complete  details.  This  latter 
approach  is  numerically  investigated  in  (36). 

The  representation  problem  is  quite  important  in 
that  the  time  between  estimates  can  be  improved  by  an 
order  of  magnitude  through  a careful  choise  of  the 
representation.  While  clearly  this  problem  of 
representation  is  important  and  deserves  careful  study, 
it  seems  to  be  a second  order  effect  relative  to 
computation  time  of  estimates,  while  the  choice  of 
synthesis  device  is  first  order.  In  a later  section 
we  will  discuss  other  synthesis  devices  which  promise 
two  or  more  orders  of  magn;tude  speed  improvement  over 
synthesis  by  third  generation  serial  digital  computers. 


4 . A Typical  Problem 


The  problem  of  phase  demodulation  is  a problem  of 
low  state  dimension  and  has  been  extensively  in- 
vestigated both  from  the  point  of  view  of  optimal  and 
suboptimal  des1gn--see  (201  for  references  to  a 
universally  used  suboptimal  desiqn,  the  phase  lock 
loop.  For  this  problem  the  following  model  is 
appropriate: 
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(4.1) 


where  un  Is  a Gaussian  white  noise  sequence  of  zero 
mean  and  variance  .‘.q.  The  initial  condition  on  (4.1) 
is  bivariate  normal  and  independent  of  the  plant  noise 
un.  The  observation  process  is 


If  the  sensor  were  linear,  the  Wiener  theory  would 
show  that  the  mean  square  error  In  estimating  phase 
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VT  q^ 
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for  continuous  observations,  which,  of  course.  Is  a 
lower  bound  on  the  mean  square  error  of  the  phase 
demodulation  problem. 

The  cyclic  loss  function,  1/2(1  - cos(x„  - x*)) 
was  considered  originally  In  (4)  and  rediscovered  in 
[ 40 ) and  still  later  In  (41),  all  In  the  context  of  a 
less  realistic  phase  demodulation  problem  where  the 
phase  Is  Brownian  motion'  , Instead  of  the  Integrated 
Brownian  motion  model  represented  by  (4.1).  This 
cyclic  loss  function  is  appropriate  for  problems  where 
one  is  interested  only  In  estimating  relative  phase. 

The  cyclic  estimate  that  xff  , which  minimizes  the 
cyclic  loss,  is  the  argument  of 

2->  2-/i 

' itv  IL  *’*  • 

where  .n(x,y)  is  the  conditional  distribution  of 

mod  2-,  /n  mod  2,/a  given  the  observations.  In  fact, 
by  consideration  nf  the  estimation  of  relative  phase, 
the  relevant  conditional  distribution  for  filtering  can 
be  taken  as  the  distribution  on  Torus,  T arising  from 
cpnditional  distribution  of  phase  and  phase  rate, 
xn  , x p given  the  observations , ,'n(x,y),  where 

■„(x.y)  * l ./r(x  * 2 v.  y » & .)  (4.3) 

u .> 

for  (x,y)  ( T.  Extensive  iionte  Carlo  simulation  of  the 
cyclic  nonlinear  filter  has  shown  that  the  cyclic 
estimate  achieves  a 3-db  error  performance  improvement 
over  the  phase  lock  loop--see  (42).  The  first  results 
w er»  CDtained  for  a single  q and  a point  mass  filter, 
example  6 of  Section  3,  and  for  each  value  of  ft, 
three  hours  of  6600  C.P.U.  were  required.  Later,  by 
using  the  Fourier  representation  of  the  density,  the 
C.P.’J.  time  was  cut  by  a factor  of  10.  Finally,  In 
(43)  we  demonstrated  that  mean  squire  error  for  the 
optimal  demodulator  was  independent  of  q.  Details  on 
the  representation,  Example  7,  Section  3.  can  be  found 
ii  1 44 ) . 

It  ts  clear  from  this  example  that,  while 
significant  error  vsrlance  reduction  Is  possible  with 
a serial  dignal  computer  as  a rezlizatlon  device,  the 
massive  comruiatlonal  task  associated  with  accurate 
synthesis  aad  Konte  Carlo  ""•or  analysis  limit  the 
state  dimension  of  the  nonlinear  filters  one  can 
effective. y build  and  analyze. 
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where  v*  are  independent  Gaussian  white  noise 

sequences  of  zero  mean  and  variance  r/A,  and  uncor- 
related with  Xo  • xj  and  un*  The  sampling  rate  A 
Is  chosen  as  . 

« ■ ->  vr  (5)  • 


on  the  basis  of  a linear  analysis  to  assure  good 
approximation  of  continuous  data— see  (21 1 and  (26). 


5 • 'lore  Iftectiye  Synthesis  Oevices 

It  became  clear  very  early  that  serial  realization 
was  effectively  speed-limited  by  the  convolution  task, 
(I.B).  necessary  to  "update"  the  a priori  conditional 
density  to  obtain  the  a posteriori  conditional  density 
for  the  state  A seconds  later,  when  a new  piece  of 
data  is  received.  From  the  struc*ure  of  (1.8),  it  is 
clear  that  immense  estimate  computation  time  reduction 
can  be  obtained  by  using  a i 'llel  digital  computer  as 
the  synthesis  tool.  An  ana'  of  the  possible 
savings  is  given  in  (29). 


'Vien  the  phase  Is  Brownian  motion,  error  variance  improvement  due  to  using  the  nonlinear  filter  Is  only 
about  .7  db,  and  further,  the  absence  of  the  necessity  of  phase  rate  tracking  makes  the  problem  of  little 
practical  Interest,  except  perhaps  for  classroom  discussion. 
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A feasib’lity  study  of  the  synthesis  of  optical 
filters  using  a hybrid  system  to  achieve  parallelism 
foi  toe  convolution  task  Is  reported  in  (<61.  This 
study  used  a serial  machine  to  simulate  a contemporary 
hybrid  system  with  MCBSSL  as  the  simulation  language. 
The  results  obtained  in  this  feasibility  study 
indicated  that  a hybrid  system  was  capable  of 
achieving  considerable  time  saving,  albeit  with  only 
t.»o  place  estimate  accuracy.  In  the  last  year,  a 
hybrid  nonlinear  filter  was  built  at  the  Labatorto 
d'Auto-atico,  University  Polytecnico  Barcelona,  Spain, 
using  an  Electronic  Associates  FAI-680  hybrid  system 
with  a floating  point  processor.  This  hybrid  non- 
linear filter  achieved  the  characteri sties  predicted 
in  [461,  <nd  the  results  are  reported  in  (45). 

Another  approach  is  using  a contemporary 
parallel  machine,  say  the  Illiac,  as  the  synthesis 
tool;  preliminary  estimates  indicate  that  three-hour 
•’note  Carlo  runs  on  the  CDC  6600  can  be  accomplished 
in  three  minutes  on  the  ! Iliac  and,  more  importantly, 
nonlinear  filters  corresponding  to  problems  with  four- 
state  dimensional  signal  orocess  models  can  be  built 
a"d  Monte  Carlo  error  analysis  perfor-ed  routinely. 
This  is  an  area  of  our  current  research  interest. 

Finally,  it  is  clear  that  special  purpose  serial 
machines  fabricated  on  acoustic -optic  or  surface  wave 
principles  in  theory  and  for  simple  signals  in 
practice  can  achieve  temporil  convolutions  in  6600 
cycle  time  about  200  ninoseconds--see  (47!.  In  |48|, 
an  approximate  homomorphism  between  the  Banach 
Algebra  fl,  of  periodic  functions  of  one  variable  and 
the  Banach  Algebra  B(.  of  functions  on  the  r- 

dimensional  torus.  The  myl tip! ication  in  thpse 
algebras  is  the  appropriate  convolution.  When  r * 2, 
then  F and  G in  B;>  , we  have 

i(F*G)  = j(F)*  «(G)  (S.l) 


where  ; is  the  appropriite  ring  homomorphism.  The 
“'•aning  of  (5.1)  is  that  (1.8)  can  b“  computed  by 
performing  a temporal  convolution  of 

i(S„)  and  :(Fn)  , 

for  the  phase  demodulation  problem  and  tne  temporal 
convolution  can  be  done  using  surface  waves  generated 
by  i C F n ) cn  a piezo-electric  crystal  with  photo- 
graphically deposited  metallic  fingers  corresponding 
to  >(Sn) . This  becomes  most  interesting  when  Sn  is 
independent  of  n as  in  the  case  of  the  phase 
demodulation  problem.  Such  a device  is  r trreitly  in 
the  planning  stage  and  is  a joint  research  project 
of  the  author  and  Or.  Fugen°  Dieulesaint  of  Lcole 
Superleur  de  Chieme  et  Pbysie,  Paris. 

6.  Conclusions 

In  this  paper  we  have  reviewed  some  of  the  first 
attempts  to  synthesize  the  optimal  nonlinear  filter. 

The  problem  itself,  while  extremely  important,  induces 
solution  methods  which  are  extremely  time  consuming 
because  of  what  Bellman  has  aptly  called  the  curse  of 
d^e  sionality.  The  current  technics,  while  primative, 
are  oplicable  to  a wide  variety  of  problems, 
including,  for  example,  the  solution  of  parabolic 
partial  differential  equations  in  more  fhan  one  space 
di“ensi on  and  are  of  importance  if  only  for  this 
application.  This  survey  will  have  served  its 
purpose  If  it  succeeds  in  interesting  research 
workers  in  pursuing  these  problems  further  and 
developing  new  method;  of  practical  synthesis. 


Per  Jrences 


R.  L.  Stratonovich,  "Conditional  Markov  Processes',' 
Theory  of  Prob.  Ai  )1 . , 5,  1960,  155-178. 

A.  N.  F.’oTiiiogorov,  T"no?r  die  Analytischen  Methoden 
in  der  Wahrschelnlichkeitsrechnung,"  Math. 
Ann.,  104,  1931,  415-458. 

K.  Ito,  "()n  Stochastic  Differential  Equations," 
Men.  Amor.  Math,  Joe..  Vol .4 , 1951. 

R.  L.  Stratonovich  Conditional  Markov  Processes 
and  Their  Application  to  CptTmal  Control 
Elsevier,  bew  York,  1966. 

H.  P.  Kushner,  "On  the  dynamical  equations  of 
conditional  probability  density  functions,  with 
applications  to  optimal  stochastic  control 
theory,"  J.  Math.  Anal.  Appl..  8.  1964. 
332-344. 

K.  Ito,  "lectures  on  stochastic  processes," 

Lecture  Notes,  Tata  Institute  of  Fundamental 
Research,  Bombay,  India,  1361. 

R.  S.  Bury,  "Nonl inear  filtering  theory,"  1 .E.E.E. 
Trans.  Automatic  Ccntr.  (corresp.),  AC- ' 0 . 1965 
198. 

R.  S.  Bucy  and  P.  D.  Joseph  Filtering  for 

stoch is tic  processes  with  applications  to 

guidance  interscience,  bew  fori  , 

R.  I.  Rortensen,  "optimal  control  o(  continuous 
time  stochastic  svstems,"  Thesis,  E.E.  Dept., 
University  of  Caliiornia,  Berkeley,  1966. 

P.  S.  Bucy,  "Linear  a'd  nonlinear  filtering," 

Proc.  I.F.F.F..,  58.  1370,  854-864. 

G.  Kallianpur  and  C.  Striebel,  "Estimation  of 
stochastic  systems,  etc."  Annals  of  Math. 

Stat. , 39,  1969,  765-301. 

A.  Eensoussan,  "Statistical  estimation  in  Hilbert 
spaces:  Application  to  Bucy's  representation 
theorem,"  Proc.  6'^  3ra que  Sym.  on  i 1 nrma_^ 
tion  Theory,  19/1 , Academia,  Prague,  19/T, 
in/- 173.” 

P.  A.  Meyer,  "Sur  un  srobleme  de  filtration," 

Seminar  de  probabilities,  Pt  VII,  Lecture  Notes 
in  Math.,  321,  'nr<n~er,  fiew  York,  197-206. 

YuSk’brokhnd  Studies  in  the  theory  of  random 
process  Addison  ;.,e  ley,  Reading  Ma;s,"T965. 

P.  A.  frost,  "Estimatici  in  continuous-time  non- 
linear systems,'  'has is , Deot.  uf  Electrical 
Engineering,  Stanford  Univ»rsitv,  1968. 

T.  Lai  lath,  “A  view  of  three  decades  of  linear 
filter  theory,"  I. E.E.E.  Trans,  on  Inform. 
Theory.  IT-20,  2,  1974,  146-180. 

Y.  C.  Ho,  R.  C.  K.  Lee,  "A  Bavesian  approach  to 
problems  in  stochastic  estimation  and  control," 
l.F.f.E.  Trans,  '-uto. Contr, . 9,  1964,  333-339. 

E.  Wong,  M.  Zakai,  !a  tne  relations  between 
ordinary  and  stochastic  differential  equations 
and  applications  to  stochastic  problems  in 
control  theory,"  free.  3rd  I F AC  Conqr. .London: 
Butterworth,  1966. 

K.  I).  Senne,  "Machine  independent  Monte  Carlo 
evaluation  of  the  performance  of  dynamic 
stochastic  systems,"  tr.ochastics . 1 , 3,  1974, 
215-233. 

R.  5.  Bucy,  A.  J.  Cal  1 inckrodt,  "An  optimal  phase 
demodulator,"  Stochastics . 1,  1_,  1973,  3-23. 

P.  S.  Bucy,  C.  Hecht.  K.  D.  Senne.  "Hew  methods 
for  nonlinear  filtering,"  Rev.  Francais  d' 

An tomatique,  J-1 , 1973,  3-54. 

R.  S.  Bucy,  '"Realization  of  nonlinear  filters," 
Proc.  2nd  Symp.  on  nonlinear  Estimation,  1971, 
Western  Periodicals. 

f-l.  Zakai  and  .1.  Ziv,  "Lower  and  upper  bounds  on 

■J  the  optimal  filtering  error  of  certain  diffu- 
sion processes,"  I. E.E.E.  Trans,  on  Inform. 
Theory,  IT-18,  l"1 72,  325-33!. 


5 


Reference*  (continued) 


|2«’]  J.  T.  Lo,  "Finite  dimensional  senior  orbits  and 
optimal  nonlinear  filtering."  l.E.E.E.  Trans, 
Inform,  Theory.  lT-lfl.  5.  1972,  583-588. 

(25)  R.  5.  BucyT  Bayes  theorem_and  digital  realiza- 

tlon  for  nonlinear  filters,"  KVI I , 2,  1969, 
60-94. 

( 26)  R.  S.  Bucy,  C.  Hecht,  K.  D.  Seme,  "An  engineer's 

guide  to  building  nonlinear  filters," 

F.  J,  Seller  Res.  I ab.  Rt.  SPL-TP-72-0004, 

0.5.  Air  Force  Ac-/ demy,  Colorado  Spring,  1 972. 

(27)  R.  S.  Pucy,  H.  Youss'f,  ‘Courier  realization  of 

the  optimal  phase  demodulator,"  Proc.  4th 
Symp.  on  Nonlinear  f stlma t i on  Theory  anf  Its 
Appl  Icatlons,  San  Diego,  "Bestern  Period iciTs, 
V97J,  34-31T 

(2d)  H.  Youssef,  "Interpolatlve  spline  filters," 

PhD. Thesis , Aerospace  Eng.  Dept.,  University 
oT^outfiern  California,  1975. 

( 29 i R.  S.  Bucy,  K.  0.  Senne,  "Oigital  synthesis  of 
nonlinear  filters,"  Autometlra,  7,  1971, 
287-298. 

(30)  R.  J.  P.  de  Fiqueiredo  and  Y.  G.  Jan,  “Spline 
filters,"  Proc.  2nd  Symp.  Nonlinear  Est. 
Theory  and  Applications,  San  Diego,  1971 , 
Western  Pe7TodicaYfTiB-99. 

(Jt)  H.  W.  Sorenson  and  D.  L.  Aslpach,  “Recursive 
Bayes  estimation  using  Gaussian  sums," 

Automat! ca.  7.  1971,  465-479. 

(32)  B.  W.  Licht.  Thesjs,  Systems  Research  Center, 

Case  Institute  of  Tech.,  Cleveland,  1970. 

(33)  G.  Kallianpur,  C.  Strlebel , "Stochastic 

differential  equations  occurring  In  the 
estimation  of  continuous  parameter  stochastic 
processes,"  Theory  Prob.  and  Appl , , 16.  5, 
1969,  597-622. 

(34)  Rene  Thom,  "Structural  stability  and  morpho- 

genesis," Benjamin,  Reading,  Hass.,  1975. 

(35)  GeraldAlengrln,  Thesis  d'F.tat,  University  Paul 

S’batier,  Toulouse,  France,  1974. 

[361  R.  Tye,  Thesis,  Dept,  of  Aerospace  Eng., 

University  of  Southern  California.  Los  Angeles 
1975. 

(37)  R.  S.  Bucy,  R.  Tye,  H.  Youssef,  "Nonlinear 

filtering  and  singular  perturbation,"  Proc. 

6th  Symp.  on  nonlinear  estimation  and  its 
Appl 1c.,  San  Diego,  1975. 


(3S)  J.  B.  Keller  and  D.  W.  McLaughlin,  "Th?  Feynman 
integral,"  Amer.  Hath.  Monthly.  82.  5.  1975. 
451-465.  “ “ 

(3t)  R.  P.  Frynman,  A.  R.Hibbs,  Quantum  Mechanics  and 
Path  Integrals.  “cGraw-Hi IT,  l/ew  York,  )94$. 

(40)  A.  J.  i!al  1 inckro'dt,  P.  S.  Bucy,  S.  Y.  Cheng, 

"Final  Peport  for  a design  study  for  an  optimal 
nonlinear  receiver/demodulator,"  NASA  Contract: 
NASS -10789  Goddard  Space  Blight  Center. 

Grer nbelt,  Maryland,  August  51,  1970. 

(41 ) A.  S.  willsky,  “Fourier  series  and  estimation, ettft 

Proc.  Inf.  Theory,  IT-20,  1974,  577-583. 

1 4 2 1 R.  S.-  Bucy.  H.  Youssef,  T5ptimal  phase  demodula* 
tion,"  submitted:  l.E.E.E.  Trans.  Autom.  Contr. 

1 43)  R.  S.  Bucy,  H.  Youssef,  "Dependence  of  the 
optimal  phase  demodulator  on  statistical 
parameters,"  l.E.E.E.  Trans.  Autom.  Contr., 
AC-20.2,  )975.-T5*35ir 

1 44 1 H.  Youssef,  “interpolative  spline  filters," 

Thesis,  Dept,  of  Aerospace  Eng.,  University  of 
Southern  California,  Los  Angeles,  Jan.  1975. 

|43)  L.  Basanez,  P.  Brunet,  R.  S.  Bucy,  R.  Huber, 

D.  S.  Miller  and  J.  Pages,  "Hybrid  nonlinear 
filters,"  submitted:  Proc.  6tn  Symp  Nonlinear 
Estimation  and  Appl.,  San  Diego,  T975T 

1 461  R.  S.  ”feuc y , M.  Herrit,  D.  S.  Hiller,  “Hybrid 
‘synthesis  of  the  optimal  discrete  nonlinear 
filter,"  Stochastics.  1.  1973,  151-211. 

( 4 / | E.  Dieulesaint,  D.  Poyer  Qndes  Elasttoue  dans 
les  So  1 i de_s ,_Avee_  applications' traitement  dp 
sinnal  Hasson,  Paris,  1974. 

) 48)  R.  $7*Iucy,  A.  J.  fallinckrodt,  H.  Youssef, 

"High  speed  convolution  of  periodic  V.Ction," 
submitted:  Siam  Jour,  Appl . Hath. 

(49)  C.  Hecht,  "Digital  realization  of  nonlinear 
filters,"  Proc.  2nd  Syirp.  on  Nonlinear 
Estimat ion~Theory  ano  Its 'Appl.  , San-?! ego, 
197T,'8h-99;  seeTTso:  thesjs*  Aerospace  Eng. 
Dept.,  University  of  Southern  California, 

Los  Angoies,  1972. 


